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Quantized  Frame  Decompositions 

M.  Craizer,  D.  A.  Fonini,  Jr.,  and  E.  A.  B.  da  Silva 


Abstract.  In  this  paper,  we  consider  a certain  type  of  decomposition 
of  vectors  in  frames,  in  which  the  coefficients  are  already  quantized  and 
thus  are  ready  for  coding.  This  decomposition  is  a generalization  for  vec- 
tors of  the  usual  binary  expansion  of  real  numbers,  and  the  algorithm  for 
obtaining  it  can  be  seen  as  a quantized  version  of  the  matching  pursuits  al- 
gorithm. We  show  that,  in  several  cases,  applying  this  algorithm  is  better 
than  first  finding  the  frame  coefficients  and  then  quantizing  them. 


§1.  Introduction 

Let  T — {ei,  C2, . . . , ep}  be  a collection  of  unit  vectors  generating  IRA.  This 
means  that  every  x € Il'v  can  be  expressed  as 

p 

x = 

i=l 

The  vectors  {ei,e2, . . . , ep}  may  or  may  not  be  linearly  independent.  In  the 
case  that  they  are  linearly  dependent,  the  set  T is  called  a frame  or  an  over- 
complete basis.  In  this  paper,  we  shall  call  T a frame  even  if  the  vectors 
{ei,e2, . . . ,ep}  are  linearly  independent.  More  on  frame  expansions  can  be 
found  in  [7]. 

Let  q = 2p  and,  for  1 < i < p,  let  V{  = e,  and  vt+p  = — e*.  We  shall  call 
the  set  V = {wi, . . . , vq}  a codebook  or  a dictionary.  Let  a be  a real  number 
in  the  interval  (0, 1).  A representation  of  a vector  x £ IRjV  in  the  form 

OO 

x = ^ alVki 
i=0 

with  Vk{  € V,  will  be  called  an  (a,  2?)-expansion.  When  the  dictionary  being 
used  is  clear  by  the  context,  we  shall  call  this  representation  simply  an  a- 
expansion.  Observe  that  the  (a,  X>)-expansion  of  a vector  x can  be  seen  as  a 
decomposition  of  x in  the  frame  T. 
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Define  the  n—  residual  of  a vector  x by 


rn(x)  = 


r o(x)  = x,  if  n = 0, 

x - E"=o!  a'vki,  if  n > 0. 


Given  x,  the  sequence  (kg,  Aq, . . .)  can  be  obtained  recursively  by  the  relation 


(rn(x),vkn)  = max  (rn(x),  vk)  - 

k 

We  shall  call  this  algorithm  the  nearest  point  algorithm  and  it  may  be  seen  as 
a quantized  version  of  the  matching  pursuits  algorithm  [4] . 

Denote  by  Aa  = Aa(V)  the  set  of  points  of  1RN  that  can  be  represented 
as  an  a-expansion  of  vectors  that  belong  to  V,  and  by  A®  = A°  (V)  the  subset 
of  Aa(V)  whose  a-expansion  can  be  obtained  by  the  nearest  point  algorithm. 
In  order  for  the  a-expansion  or  the  nearest  point  algorithm  to  be  a suitable 
scheme  for  quantized  frame  decomposition,  we  must  choose  a such  that  AQ 
or  A° , respectively,  contain  an  open  set  of  K'v . In  Section  2,  we  shall  give 
conditions  on  a that  guarantee  these  facts. 

At  this  point,  a question  arises:  is  it  worthwhile  to  decompose  a vector 
in  a frame  using  the  a-expansion,  or  is  it  better  to  decompose  it  in  the  usual 
way  and  then  quantize  the  coefficients  in  a second  step  (see  [6,9])?  We  shall 
answer  this  question  by  considering  the  rate-distortion  characteristic  of  each 
scheme.  We  show  in  Section  3 that  the  first  scheme  is  better,  in  an  asymptotic 
sense,  if  and  only  if  we  can  choose  a satisfying 

!2i1W< 

l0S2  5 

We  shall  also  give  examples  where  this  inequality  holds. 

Take  x = (xj, . . . , x/v)  6 We  can  quantize  x by  taking  the  n-term 
binary  representation  of  each  coordinate  x; . This  procedure  can  be  considered 
as  an  n-term  a-expansion  using  the  dictionary  Bn  whose  code  vectors  are 
the  corners  of  the  hypercube  [— 1,1]  ^ and  a = |.  So  the  a-expansion  in  an 
arbitrary  dictionary  V can  be  considered  as  a generalization  of  the  usual  binary 
expansion  for  vectors.  The  relevant  question  is  whether  there  is  any  dictionary 
V that  is  better  in  some  sense  than  Bn-  It  is  worthy  of  note  that  some  special 
dictionaries,  related  to  the  sphere  packing  problem  [2],  have  already  been  used 
for  image  coding,  yielding  better  results  than  Bn  ([3,8]). 


§2.  Theory  of  Alpha-Expansions 
General  representation 

Let  V — vq}  be  a collection  of  vectors  that  generates  all  IiN , and 

0 < a < 1 be  a parameter.  Denote  by  Aa  the  set  of  points  x 6 RA'  that  can 
be  written  as 

OO 

x - y Za*vki 

t=0 
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with  Vk{  € V.  Let  Pa  be  the  convex  hull  of  the  vectors  {jz^;vk}k=i,...,q- 
Observe  that  any  i 6 A„  is  a linear  combination  of  the  vectors  {vp,}k=it..,q 
with  coefficients  whose  sum  is  not  larger  than  yA— , and  therefore  Aa  C Pa. 

Define  the  contracting  maps  fk  = fk,a  by  /*,( x)  = ax  + for  k € 
{1, . . . ,q}.  We  observe  that  /*.  is  a homotety  of  center  which  implies 

that  fk(Pa)  C Pa-  Therefore  the  set  {/i, . . . , fq}  forms  an  iterated  function 
system  (IFS)  [1]  on  Pa.  It  is  not  difficult  to  show  that  the  attractor  of  this 
system  is  exactly  Aa,  i.e., 


Aa = n Fn(pa), 

n= 0 


where  F = Fa  is  the  function  of  sets  defined  by 

F(A)  = f1(A)U---Ufq(A). 


Example  1.  Let  V = Bn,  the  dictionary  whose  code  vectors  are  the  corners 
of  the  hypercube  [ — 1, 1]JV.  For  any  a > we  have  that  F(Pa)  = Pa  and 
therefore  Aa  = Pa. 

We  are  interested  in  finding  the  smallest  value  of  a such  that  Aa  contains 
an  open  set.  In  the  above  example,  this  occurs  for  a = \,  when  in  fact 
Aa  = Pa. 

Remark  1 . One  can  show  that  for  any  dictionary,  if  a > , then  Aa  = Pa, 

which  shows  that  the  smallest  value  of  a such  that  Aa  contains  an  open  set 
is  smaller  than  jyyy  ■ 

In  Example  1,  the  smallest  a such  that  Aa  contains  an  open  set  satisfies 
also  Aa  — Pa-  But  this  is  not  a general  fact,  as  the  following  example  shows. 

Example  2.  Let  V = B3  U {(1, 0, 0),  (— 1, 0, 0)},  where  B$  is  the  dictionary 
whose  code  vectors  are  the  corners  of  the  cube  [— 1,  l]3.  If  we  consider  a = 
~,  then  [ — 1,  l]3  C Aa,  but  Aa  is  strictly  contained  in  Pa.  This  fact  can 
be  seen  by  observing  that  the  centroid  of  the  face  of  Pa  whose  vertices  are 
(1,0,0),  ^(1,1,1)  and  ^j(l,  1,  —1)  are  not  contained  in  F(Pa),  which  implies 
that  F(Pa)  ^ pat  and  thus  Aa  ft  pa. 

In  all  examples  that  we  have  considered,  we  observe  that  if  Aa  contains 
an  open  set  of  1RA  , then  it  also  contains  the  convex  hull  of  some  of  the  points 
of  the  dictionary.  We  don’t  know  whether  this  is  always  true,  so  we  formulate 
it  as  a question: 

Question  1.  If,  for  some  0 < a < 1,  Aa(V)  contains  an  open  set  of  1R  V,  then 
will  a subdictionary  V\  C V always  exist  such  that  Aa(V)  D Pa (Vi ) ? 
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Basic  algorithm 

How  do  we  obtain  the  sequence  of  indexes  (ko,  fci,  • • •)  that  represent  a given 
vector  x £ Aa?  In  general,  the  representation  of  a vector  x is  not  unique.  In 
order  to  define  which  of  the  sequences  representing  the  vector  x we  shall  look 
for,  we  consider  a choice  function  K : F(Pa)  — » {1, ...  ,9}  with  the  following 
properties: 

1)  A 04)  C 14,  for  any  k £ {l,...,g},  where  Vk  = K~1(k). 

2)  If  K(x)  = k,  then  x £ fk{Pc, ),  for  any  x £ F(Pa). 

It  can  be  shown  that  such  a function  always  exists.  This  choice  function  K 
determines  a function  g : F(Pa ) — > Pa  given  by 


9(x)  = 


x ~ VK(X) 
a 


It  is  not  difficult  to  show  that 


a a = n~  ig-n(Pa). 

This  implies  that  if  a:  € Aa,  then  g'(x)  £ Aa,  for  any  i > 0. 

By  the  last  paragraph,  given  x £ Aa,  we  can  choose  the  sequence 
(ko,ki,...)  by  the  relation  kt  = K(g'(x)).  We  shall  call  this  the  basic  al- 
gorithm. This  algorithm  always  works,  but  it  is  computationally  expensive. 
So  we  propose  another  algorithm,  computationally  feasible,  called  the  nearest 
point  algorithm. 


Nearest  point  algorithm 


The  nearest  point  algorithm  is  used  for  obtaining  a sequence  ( ko , k\, . . .)  repre- 
senting a vector  * £ Aa . It  can  be  seen  as  a quantized  version  of  the  matching 
pursuits  algorithm. 

It  is  determined  by  the  choice  function  Ko  defined  by  the  property  that 
Ko{x)  is  the  code  vector  in  V nearest  to  x.  We  denote  by  Vo.it  the  set  KQl(k) 
and  by  go  the  function 


9o{x)  = 


x - VKo(x) 


This  choice  function  certainly  satisfies  Property  1 above,  but  Property  2 can 
fail.  It  is  not  difficult  to  see  that  Property  2 holds  if  and  only  if  Vo ,k  C fk{Pa), 
for  every  k £ {1, . . . , q}. 

Let  A°(2?)  = n“=1s^n(PQ)-  We  have  that  A°(P)  C Aa(V),  but  they  are 
not  necessarily  equal.  One  can  verify  that  A°a(V)  = Aa(V)  if  and  only  if  the 
choice  function  Kq  satisfies  Property  2 above. 


Example  3.  LetV  = {(1, 0),  (0, 1),  (-1, 0)}.  For  any  0 < a < 1,  the  segment 
(0,  (5),  0 < (5  < 1 is  not  contained  in  AC-Pa)-  Therefore  go(0,  A)  is  not  in  P„, 
which  implies  that  this  segment  is  not  in  A^(V).  On  the  other  hand,  if  a > |, 
A a{T>)  = Pa- 

We  have  also  observed  in  examples  that  if  A°  (V)  contains  an  open  set  of 
IRa  , then  it  must  contain  the  convex  hull  of  some  code  vectors.  This  prompts 
the  following  question: 
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Question  2.  If,  for  some  0 < a < 1,  (V)  contains  an  open  set  of  JRN , then 

will  there  be  a subdictionary  V i C~D  such  that  A°  (V)  D PQ(D1)? 


§3.  Comparison  between  Alpha-Expansions  and  the  Decompose- 
Quantize  Procedure 

In  this  section  we  shall  compare  the  a-expansion  in  a frame  with  the  2- 
step  procedure  of  first  decomposing  in  the  frame  and  then  quantizing  the 
coefficients  so  obtained  in  a second  step.  We  shall  do  this  by  comparing  the 
rate-distortion  functions  of  each  scheme. 

Let  T = {ei,  e2, . . . , ep}  be  a frame  in  IR^,  and  take  x € JRW  with  ||a;||  < 
M.  We  shall  assume  that  the  coefficients  (ai,  <12, . . . , ap)  of  the  decomposition 
x in  the  frame  T satisfy  |dj|  < C\M,  for  some  constant  Cj  that  depends  only 
on  T . We  shall  consider  here  the  quantization  of  these  coefficients  by  binary 
expansions.  If  each  coefficient  is  represented  by  n bits,  the  total  number  of 
bits  used  is  R = np,  and  the  maximum  square  error  per  coefficient  is  given 

by  \C\M  (I)"  XJ  . If  we  multiply  this  by  p,  we  obtain  the  total  maximum 
square  distortion  D.  Therefore,  we  can  write  the  rate-distortion  relation 


Let  us  consider  now  the  a-expansion  procedure.  If  we  approximate  x € 
Aa,  INI  < M,  by  its  n-term  a-expansion  (vi0,Vj1, . . . ,Vin_1),  the  maximum 
square  distortion  is  given  by  D = [ C^Ma” ]2  , where  C2  is  a constant  that 
depends  only  on  T . The  number  of  bits  necessary  to  code  this  sequence  is 
R = n log2(2p),  and  thus  we  have  the  rate-distortion  relation 


R = 


log2  (2p) 

21oS2(^) 


log2 


We  conclude  that  asymptotically,  the  a-expansion  is  better  than  the 
decompose-quantize  procedure  if  we  can  choose  a such  that  AQ  contains  an 
open  set  and 

< P 

Example  4.  Let  F = |)i(1j0),(—  and  take  a = \.  Then 

Aa  = Pa  (which  in  this  case  is  a hexagon ) and 


logz  (2P) 
loS2  (5) 


log2(2P) 

1oS2  i 


= log2  6 < 3, 


which  implies  that  in  this  case  the  a-expansion  is  better  than  the  decompose- 
quantize  procedure. 


158 


M.  Craizer,  D.  A.  Fonini,  and  E.  A.  B.  da  Silva 


x x nearest  point  algorithm 

* * matching  pursuits 


-a 


0.5  1 1.5  2 2.5  3 

Rate  in  bits  per  source  sample 


Fig.  1.  The  8-dimensional  hypercube  as  codebook. 


Remark  2.  If  for  a given  frame  T , Aa  contains  an  open  set,  then  this  remains 
true  for  any  other  frame  T\  obtained  from  T by  adding  some  more  vectors. 
Hence,  even  if  relation  (1)  does  not  hold  for  T,  it  will  hold  for  highly  redundant 
frames  containing  T . 

Remark  3.  By  Remark  1,  a > implies  that  A„  = Pn.  Therefore,  if  we 
consider  frames  satisfying 


log2  (2p) 


< log2 


N + 1 


P ~ \ N 

the  a-expansion  scheme  will  be  better  than  the  decompose-quantize  procedure. 


Experimental  results 

In  order  to  directly  compare  our  method  (the  nearest  point  algorithm)  with 
some  established  results,  we  look  at  some  examples  presented  in  [6]  (3.4.2,  pp. 
41-45).  To  this  end,  we  used  a similar  source  and  the  same  codebook. 

A zero-mean  gaussian  AR  source  with  correlation  coefficient  p = 0.9  was 
used  to  provide  the  data  points.  Vectors  were  formed  by  blocks  of  N samples. 
Rate  was  measured  as  the  first  order  entropy  of  the  index  stream  produced 
by  the  algorithm  - similarly  to  [6]. 

In  Fig.  1,  we  used  the  vertices  of  the  8-dimensional  hypercube  as  the 
codebook.  In  this  case,  a was  set  to  0.501  and  ||  v*  ||  = 1.4142  (that  is  ^). 

As  can  be  seen  from  this  data,  our  method  does  give  some  performance 
benefits  on  low  bit-rates. 

Although  our  algorithm  can  be  very  expensive  in  terms  of  computational 
effort,  so  are  other  greedy  algorithms  like  matching  pursuits.  But  there  are 
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some  well  structured  codebooks  which  lend  themselves  to  fast  calculation  of 
the  steps  involved  - like  the  one  used  in  this  example. 


§4.  Conclusions 

In  this  paper  we  have  further  developed  the  theory  of  a-expansions  and  applied 
it  in  the  context  of  quantized  frame  expansions. 

We  have  shown  that  a-expansions  perform  asymptotically  better,  in  a 
rate-distortion  sense,  than  the  decompose-quantize  method.  In  addition,  pre- 
liminary experimental  results  indicate  that  this  method  also  compares  favor- 
ably to  the  decompose-quantize  method  in  practical  cases.  This  was  verified 
by  direct  comparison  between  our  method  and  quantized  matching  pursuits 
from  [6]. 
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